A Method for Speeding Up Value Iteration in Partially Observable Markov Decision Processes

We present a technique for speeding up the convergence of value iteration for par­ tially observable Markov decisions processes (POMDPs). The underlying idea is similar to that behind modified policy iteration for fully observable Markov decision processes (MDPs). The technique can be easily incor­ porated into any existing POMDP value it­ eration algorithms. Experiments have been conducted on several test problems with one POMDP value iteration algorithm called in­ cremental pruning. We find that the tech­ nique can make incremental pruning run sev­ eral orders of magnitude faster.